Papers with large neural network predictions
Weight Distillation: Transferring the Knowledge in Neural Network Parameters (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge distillation is an effective method for model acceleration and compression. |
| Approach: | They propose to use parameters to distill knowledge from large neural networks to small ones . they propose to do this by using a parameter generator to transfer the knowledge to a small neural network . |
| Outcome: | The proposed method learns a small network 1.88 2.94x faster than the large network but with competitive BLEU points. |